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WAVELET  SHRINKAGE 


>  Introduction  to  Wavelet  Shrinkage  and  Denoising 

>  Procedure  for  wavelet  shrinkage 
>Why  Wavelet  Shrinkage  Works 
>Two  methods  of  Wavelet  Shrinkage 


Introduction: 

□  An  image  is  often  corrupted  by  noise  in 
its  acquisition  and  transmission  stage. 

□  Noises  are  normally  created  when 
scanning  images  to  produce  digital 
images,  recording  a  voice  to  an  audio 
file,  and  even  transmitting  digital  image 
often  produce  Noise. 

□  This  noise  can  be  random  or  white 
noise  with  no  coherence  or  coherent 
noise  introduced  by  device  mechanism 


Introduction 


®  Wavelet  shrinkage  is  a  signal  denoising  technique 
based  on  the  idea  of  thresholding  the  wavelet 
coefficients. 

®  Denoising  is  the  process  of  removing  noise  from  a 
signal. 

®  Wavelet  coefficients  having  small  absolute  values 
are  considered  to  encode  very  fine  details  of  the 
signal. 

®  Wavelet  shrinkage  denoising  should  not  be 
confused  with  smoothing ,  Whereas  smoothing 
removes  high  frequencies  and  retains  low  ones, 
denoising  attempts  to  remove  whatever  noise  is 
present  and  retain  whatever  signal  is  present 
regardless  of  the  signal's  frequency  content . 


Noisy  Image 


Denoised  Image 


Threshold  rules 


There  are  two  types  of  rules  that  we  are  ging  to  use  in 
this  talk. 

Hard  thresholding  rule  -  Let  A  =  {x  :  |  x  |  >  >.} 

HiM  =  x  '  XaM 

where  %A  is  the  characteristic  set  function. 

Soft  thresholding  rule  - 

T\(x)  =  sgn(x)  •  max(  |  x  |  -  X,  0) 


Procedure  for  wavelet  Shrinkage. 

The  wavelet  shrinkage  method  involves  the  following 
steps: 

1.  Apply  wavelet  transform  to  the  signal. 

2.  Obtain  a  threshold  Value  that  minimizes  the  Mean 
Square  Error.  Then  using  the  Soft  threshold  function 
we  remove  (zero  out)  the  coefficients  that  are 
smaller  than  the  threshold. 

3.  Reconstruct  the  signal  (apply  the  inverse  of  the 
Wavelet  transform). 


Why  does  Wavelet  Shrinkage  work? 

Suppose  we  take  one  iteration  of  wavelet  transformation  to  a  noisy 
signal  to  get:  w  =  WX( t)  =  WS  (t)  +  WN(t). 

WX(t)  is  the  signal  that  contains  the  lowpass  portion  of 
transformation  in  the  first  half  and  the  highpass  portion 
transformation  in  the  second  half. 

The  highpass  portion  of  the  signal  is  sparse  since  most  of  the  energy 
is  stored  in  the  lowpass  portion  of  the  transformation. 

However,  WN  (t)  is  the  Gaussian  white  noise,  when  we  apply  an 
Orthogonal  Matrix  FT  to  a  Gaussian  white  noise  signal;  it  returns  a 
Gaussian  white  noise  signal  with  the  same  variance. 

So,  the  noise  level  for  WN  (t)  is  the  same  as  the  noise  level  for  N  (t). 
This  tells  us  that  the  highpass  portion  of  WX  (t)  is  comprised 
primarily  of  noise! 


The  observed  noisy  data  will  have  the  form 

X(t)  =  S(t)  +  N(t) 

Where  S(f)  is  the  true  signal  and  N(t)  is  the  noise  as  functions 
of  time  t  (for  sampled  values). 

We  calculate  the  k  iterations  of  the  wavelet  transformation  of 
X(f)  to  obtain  Y  =  [l  \  h],  where  1  is  the  lowpass 
(approximation)  portion  and  h  is  the  highpass  (details) 
Portion  of  Y. 

Apply  the  soft  threshold  rule  to  h  to  either  shrink  or  zero  the 
coefficients  in  h.  Let  this  new  highpass  portion  be  h'. 

Form  a  modified  transform  Y'  =  [l  \  h']  by  rejoining  /  with  h'. 
Finally,  compute  k  iterations  of  the  inverse  wavelet 
transformation  of  Y'  to  obtain  a  denoised  estimate  S'  of  S. 


Measuring  the  Effectiveness  of  Denoising 
method. 


We  can  use  the  mean  squared  error  to  determine  if  the 
shrinkage  method  does  a  good  job  of  denoising  the 
given  signal. 

The  mean  squared  error  or  MSE  is  defined  as 

/  V  N 

E  IIS-SH2 )  =  2>*-St)2 

k= 1 

We  will  construct  the  thresholds  (tolerance)  values  that 
minimizes  the  MSE  value. 


Two  methods  of  wavelet  shrinkage: 


s  Goal:  To  determine  a  threshold  (or  tolerence) 
value  X  that  minimizes  MSE  (mean  squared 
Error) 

s  There  are  two  methods  that  we  will  discuss  for 
choosing  X  that  is  the  visushrink  and  sureshrink. 

s  If  the  image  is  sparse  we  use  visushrink  method 
to  select  otherwise  we  use  sureshrink 


Sure-shrinkage 

Btein’s Wibiased ^isk  II  stimator] 

Theorem:  Suppose  the  N- vector  w  is  formed  by  w  =  z  +  e,  where 
e  =  (e,,. .  .,e  )  and  each  ek  is  normally  distributed  with  mean  theta  and 

/V  /V 

variance  1 .  Let  z  be  the  estimator  formed  by  Z  =  w  +  g(w)  where  the 
coordinate  functions  gk:  RN  arrow  R  of  the  vector-valued  function 
g:  RN  ->  RN  are  differentiable  except  at  a  finite  number  of  points.  We 
obtain 


E( 


z  —  z 


)  =  E(N  +  g(w ) 


N  3 

+  2  Y.—gk(w)) 
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~ida>, 
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To  get  the  threshold  function  we  simplify  right  hand  side 
of  equation,  then  take  the  minimum  value  to  obtain  the 
threshold 


N 


fW  =  N  +  ||g(w)||  +2^ 


a 


k 


ddco. 


gt(w) 


To  get  the  threshold  function  we  simplify  right  hand  side  of 
equation,  then  take  the  minimum  value  to  obtain  the 
threshold  .  Put 

f(A)  =  A/r  +  |g(<y)|  +  2^- — gk(co) 

k= 1  uCDfr 

By  considering  different  cases  for  we  get 

N 

f  (/l)  =  N  +  ^  min(/l2 ,  cok 2 )  -  2  #{k:  o)k  <X] 

k= 1 


Putting  CD  in  place  of  and  simplifying  we  obtain 


/(h+i|)=/(|®/|)+t^-0(®/+l2-®(2)  for  /> i 

/(|©1|)  =  iV  +  A?|®1|2 

Finally,  we  obtain  the  threshold  values  using 

/l”re  =  min{/(U|),/(M), . ,/(K|)} 


Vi  su- shrinkage 

Donoho  and  Johnston  proved  the  following  result: 


let  ’  e  /  and  given  X  =  S  +  N,  where  N  is  the 
white  Gaussian  noise,  with  noise  level  o  .  Let 
be  a  soft  threshold  function  with  =  cr^/2  ln(j  . 

/V 

If  is  a  vector  formed  by  applying  t  to  S,  then 


N 

2 )  <  (2  ln(A0  +  l)(cr2  +  min(^2 ,  cr2 )) 


We  call  the  tolerance  value  l  =  cr^J 2  ln(  to  be 
universal  threshold  and  donate  it  by 

For  and  any  S  the  soft  threshold  rule  produces 
a  mean  square  error  that  is  bounded  above  by  a 
constant  times  the  noise  level  square  plus  the  ideal 
mean  square  error. 


Notice  that  the  threshold  value  depends  on  signal  size  and 
the  noise  level  a.  In  practice,  we  do  not  know  the  value  of  a. 


To  estimate  the  noise  level  a,  we  will  make  use  of  a  result 
by  Frank  R  Hampel  that  showed  the  Median  Absolute 
Deviation  MAD(X)  =  \  X  -  Median(X  converges  to  0.6745a 

as  the  sample  size  goes  to  infinity. 

To  estimate  noise  level  o  we  use: 

&  =  MAD(h)/< 0.6745 

Where  h  is  the  highpass  portion  of  the  transformation  1st 
iteration. 


